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ABSTRACT 

Co rr espondence , survey, and group interview data were 
combined to investigate the nature and consequences of participatory 
evaluation activities in a medium-sized school district (48,000 
students) in Simcoe County in Ontario (Canada). The study focused on 
the dynamics, supporting conditions, and consequences of internally 
conducted participatory evaluation. Data were collected from staff 
who participated in several evaluation studies, including evaluations 
of student guidance services, library services, intermediate 
mathematics, intermediate history and contemporary studies, and 
French as a second language. In each case, the author, as the sole 
district staff member in a research capacity, provided technical 
support for problem format ion, instrument development , data 
processing, analysis and interpretation, development of suggested 
courses of action, and assistance in report writing. Results inform 
the researchers' understanding of the conditions supporting effective 
internal participatory evaluation. (Author/SLD) 
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Abstract 

This study combined written correspondence, survey and group interview data to 
investigate the nature and consequences of participatory evaluation activities in 
a medium-sized school district (48 000 students). The author is the sole district 
staff person in a research capacity and necessarily works closely on program 
evaluation projects with school and district office-based personnel The study 
focused on the dynamics, supporting conditions and consequences of internally 
conducted participatory evaluation. Data were collected from staff who 
participated in several evaluation studies including evaluations of guidance 
(student) services, library services, intermediate mathematics, intermediate history 
and contemporary studies, and French as a Second Language. In each case, 
technical support was provided by the author for problem formulation, instrument 
development, data processing, analysis and interpretation, the development of 
suggested courses of action based on the findings and assistance in report writing. 
The results inform our understanding of the conditions supporting effective 
internal participatory evaluation. 



A PARTICIPATORY APPROACH TO DISTRICT-LEVEL PROGRAM 
EVALUATION: THE DYNAMICS OF INTERNAL EVALUATION 



Background and Introduction 

For the past several yean I have been the sole educational researcher for a 
medium-sized school district in one of the largest counties in the province of 
Ontario, containing approximately 4 800 square kilometres of territory. The 
school district currently has approximately 48 000 students and 3 100 teaching 
staff in 82 elementary and 15 secondary schools. The 1992 expenditure budget 
was $290 300 000. 

During the 1980s a major program focus for all school systems in Ontario 
involved the implementation of provincial curriculum guidelines. The ensuing 
review, development and implementation cycle (Leithwood, 1987) resulted in a 
more visible role for program evaluation. 

Consequently, a model for conducting program evaluation was developed to guide 
internal evaluation activities in this school district (Lafleur, 1990). Evaluation 
is viewed as having an accountability component and a formative component that 
inform decision making. One feature of the program evaluation model is to 
engage primary users in as many phases of the evaluation process as possible. 
For example, all internal evaluations rely on an evaluation team that involves 
members in the n nuts and bolts n of the problem formulation, instrument design 
or selection, data collection, analysis, interpretation and reporting. Members of 
this team typically include primary users such as senior administrators, program 
support staff, school administrators, teachers and the school district researcher. 
In addition, a variety of primary users are often involved in specific phases of the 
evaluation process such as data collection, analysis and interpretation of the 
findings. 



The current focus on educational change (Fullan, 1991), school improvement 
(Stoll and Fink, 1992) and restructuring (Murphy, 1991), as well as the 
continuing public concern for accountability, ensures a continued role for 
program evaluation in this school district. In the current economic climate, it is 
increasingly obvious that expectations will continue in a context of very limited 
resources. Internal evaluation such as that described by Love (1991) and Mayne 
(1992) emphasizes working within the organizational context but also highlights 
the organization's responsibility. Consequently, the present research study 
provides data for critically reflecting on existing program evaluation in this school 
district. 

This research study acknowledges the substantial review of the literature that was 
recently completed by Cousins and Earl (1992). In fact the design of the present 
research study uses many of their ideas and deliberately operationalizes a number 
of their key concepts. 

The emphasis given by Cousins and Earl to organizational learning seems to make 
a great deal of sense in the current context of educational change and 
restructuring. Bolman and Deal (1991) and Schein (1991), for example, have 
provided extensive documentation that helps us examine and consider some of the 
issues related to organizational cultures. It seems reasonable, for example, to 
conclude that the way organizations do business influences the form and impact 
of program evaluation. 

r The power relations and existing structures are the practical everyday realities 
that may or may not match the stated beliefs and values of the organization. It 
may be critical, as Shavelson (1988) suggests, to try and understand the mind 
frames of policymakers and practitioners. By understanding better the 
perceptions and demands on policymakers the likelihood of improving evaluation 
and the utilization of findings may be increased. 



Greene (1992) also focuses on the relation between participatory evaluation and 
the organization. She suggests, however, that "the more difficult challenge ... is 
just how a collaborative inquiry process can catalyze and develop the structural 
capacity to act. n (p. 7) Weiss ' views that evaluation is a political act and that 
decision making is not always a rational enterprise also challenge evaluators to 
critically consider how they can best assist individuals in participating more 
wisely (1992, 1988). 

A case study by Pugh (1990) illustrates the difficulties of evaluation utilization. 
In spite of doing quality work, Pugh believes that "... significant findings and/or 
policy implications are not acted upon due to the combination of economic 
considerations, political realities and pi blic relations impact". 

Cousins and Earl (1992) have provided an excellent synthesis of the literature on 
participatory evaluation and evaluation utilization. Their work occurs at a 
critical time in the evolution of the evaluation enterprise. It acknowledges the 
increasing democratization of evaluation by respecting the role and responsibility 
of the individual and the organization. In addition, evaluation is growing up in 
an environment that is more tolerant, accepting and even demanding of 
methodological diversity. As we deal with new and emerging educational 
paradigms, the challenges to the evaluation enterprise will continue. 

Method 

The present research study was designed as a retrospective examination of this 
school district's approach to program evaluation In particular the study set out 
to examine the relation between the participatory nature of the evaluation process 
and the utilization of the findings. It was also designed to provide information 
about how well the current model for internal program evaluation is being 
implemented in a medium-sized school district where the author is the sole person 



employed in a research capacity. In addition, information was required about the 
strengths and weaknesses of current evaluation practices and needed changes. 

This research study collected data on the perspectives of primary users who were 
involved in a number of internal evaluations that were completed over the past 
several years. The actual research activities, including the development of 
instruments, ad ninistration and collection of data, the analysis of data, the 
interpretation of findings and the writing of the report were coordinated and 
undertaken by the author. 

In order to obtain the most reliable and valid information related to the impact 
of the participatory evaluation approach, complementary research approaches 
were used. Initially data were obtained from written correspondence initiated by 
the author with seven primary users who had participated in previous school 
district evaluations. Questionnaires were then used to collect further information 
from a wider range of primary users about their involvement in previous school 
district evaluations. Finally, a focused group discussion was undertaken with the 
original cadre of primary users who had participated in the previous two data 
collection methods. For the purposes of this study the total number of primary 
users represented about one-third of those who had previously participated as 
members of the different internal program evaluation teams over the past several 
years. 

The remaining part of the method section describes the techniques associated with 
each of the different approaches that were used in the study. 

Method 1: Written Correspondence 
The intent of the written correspondence was to gather initial information and at 
the same time involve a small cadre of individuals in the present retroactive 
research study. This method was chosen as an unobtrusive way of encouraging 
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a small select group of individuals to reflect on their involvement in previous 
program evaluations and to reconstruct their own version of significant features 
of their participation. 

Although eight persons were initially approached and agreed to participate, other 
commitments precluded one person from participating in the written 
correspondence phase of the study. Consequently, the author corresponded with 
seven individuals. In two instances one written exchange occurred. In the 
remaining five cases two letters were passed back and forth. 

Following a period of about six weeks the written correspondence phase of the 
study was concluded. Replies were then analyzed for emerging themes and issues. 

Method 2: Questionnaires 
The use of questionnaires is one way of collecting data from a large number of 
individuals in a relatively short time span. In this research study, questionnaires 
were used to focus on the perceptions of a selected group of primary users who 
had been involved in previous program evaluations. Every third person on a 
master list of all individuals who had been members of previous internal 
evaluation teams, including those who participated in the written correspondence 
phase of this study, were asked to complete a questionnaire. The group was 
selected to represent nine internal evaluations that had been completed in the 
school district within the last eight years. The focus of these evaluations included 
technical education, behaviour, grade 4-6 mathematics, ~Trench-as-a-second 
language, special education identification and placement, guidance (student) 
services, library services, grade 7-10 mathematics, and grade 7-10 history and 
contemporary studies. 

The questionnaire items were directly related to the participatory approach to 
evaluation and the utilization of the findings. The recently published article by 
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Cousins and Earl (1992) was particularly instrumental in informing the design of 
the questionnaire. For example, there was a series of questions dealing with the 
quality of the evaluation and factors affecting utilization. In addition, there were 
questions containing a number ofLikert items dealing with the views of primary 
users on the participator evaluation process, their own involvement in the 
evaluation activities, and the organizational culture of the school district relative 
to evaluation. 

Vie questionnaires were developed and responses analyzed by the school district 
researcher. Twenty-eight individuals were given questionnaires to complete. 
Although 27 completed returns were eventually received, only 24 responses were 
available when the preliminary analyses were undertaken. 

Descriptive statistics and comments were compiled for all questions. Correlation 
matrices were then calculated for most items in the questionnaire. In addition, 
factor analyses were completed for items contained in three questions -the review 
process, primary user involvement and organizational culture. Finally, regression 
analysis was used to examine the relations among a few key factors that had been 
identified in the literature, the written correspondence, descriptive analyses of 
questionnaire responses and during focused group discussion. 

Method 3: Focused Group Discussion 
Within two weeks of the questionnaires being distributed, the original group of 
eight persons who had agreed to participate in the written Correspondence were 
invited to participate in a focused group discussion. All eight individuals spent 
approximately 90 minutes sharing their ideas about the value and effect of 
internal program evaluation. Field notes were taken and summaries made of key 
ideas and suggestions. 
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Findings 



The findings are presented under three headings, each corresponding to the 
approaches described in the method section. No attempt has been made to 
provide an exhaustive analysis of the results at this time. Rather the initial 
findings and preliminary analyses are highlighted. 

Findings 1: Written Correspondence 
The written correspondence provided a unique opportunity for several primary 
users to reflect upon their involvement in previous evaluations. For some, this 
meant revisiting activities that happened up to eight years ago. Others were more 
fortunate in that their involvement occurred within the last few years. 

Each person identified their experience as a positive and worthwhile one. All 
indicated that they had learned more about the inquiry process and the program 
in question. On the other hand, most were guarded in their assessment about the 
actual impact of the evaluation, indicating limited success in the development and 
implementation of the resulting action plan. 

In order to provide a primary user perspective, a number of extracts from the 
letters are included here. Many of the issues mentioned in the literature, as 
discussed by Cousins and Earl (1992), are represented in the following comments. 



7 

10 



"Since the beginning, the researcher has been closely connected to 
program personnel Initially this occurred formally when the researcher 
had coordinating responsibilities for some subject areas ... Also the 
researcher met regularly with other program department members ... As 
such research has never been outside program. Rather it has been 
integral to it. For example, the Document implementation Plan which 
guides our system has program evaluation as pan of the process .... " 

"An evaluation steering committee was struck to oversee this project. 
Although the researcher did not chair the committee, the membership 
looked to him for leadership and direction .... Throughout the project the 
researcher laid out the blueprint ... leading us through the process .... 
In summary, the term collaborative effort might be used to describe the 
relationship. " 

"Whether or not it was the nature of the (second evaluation) project ,.. 
there was less direct involvement by the researcher. In spite of the more 
"arms length " approach by the researcher this should not be interpreted 
as a less valued contribution. The researcher was a valued consultant to 
the planning committee .... the relationship between myself and the 
researcher was different .... The difference would be in the roles and 
responsibilities .... I feel that the "training" I received in the (first 
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evaluation) project allowed me, consciously or otherwise, to rely less 
heavily on the researcher. " 

7 think that our problem is that evaluations are foreign to us. Teaching 
has largely been a very private endeavour between teacher and student. 
There has been little monitoring, supervision and dialogue even within the 
same school " 

"Much of the time in the beginning stages was spent formulating the 
criteria/questions for the evaluation .... After each meeting our work 
would be scrutinized by the Superintendent of Program. Areas that were 
deemed by him as not applicable to the evaluation were dropped. Other 
than that the evaluation team set its own goals for the evaluation. " 

"As we continued into the next phase of our task it seemed to me that we 
were working cohesively .... We would bring to each committee meeting 
our ideas and share them with the group. Ideas were accepted, 
challenged or modified in a very open manner .... Msing consensus the 
group would agree, approve, revise .... w 

"As a committee we were very proud of the work we had done, the interest 
we had generated and the suggested courses of action that we had 



developed. (What happened next) was to be our biggest disappointment 
of the whole process .... Delay after delay ensued .... No indication as 
to where our report would go was given to us .... Letters by various 
members of the committee inquiring into the status of the evaluation or 
release of the evaluation to the system were not answered. " 

"In both evaluations, for different reasons, the system did not take much 
action to put in specific items from the evaluation. The two evaluations 
were started for purposes that did not fit the realities of political 
commitment. Generally speaking, the evaluations had a fairly large 
agenda to "sharpen up " some aspects of the system. Additional, but 
minor, agenda items were to gather information and to give future 
direction.... " 

"One of the major strengths was the gathering of a variety of data in 
diverse ways. The information diversity gave the impression (an accurate 
one) that this was a very thorough evaluation covering a wide range of 
areas and leading to a workable action plan. The mix of people on the 
committee ... and the fact that the committee was chaired by a secondary 
school principal, gave a high degree of credibility to the group and led to 
strong feelings of team membership. The fact that we also had a high 
degree of credibility at the provincial level was a positive factor for 
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everyone involved. " 



"Some of the very positive things that came out of the evaluation ... were 
never publicized or brought forward in any way. That (particular) 
information was a very pleasant surprise for many of us working on the 
evaluation team and the kind of information that needed to be conveyed to 
the system. " 

"... the first thing that comes to mind is the connection I was able to make 
with (certain evaluation team members). " 

"When I think about the action plan developed after the evaluation and the 
changes that have been made because of it, Vma little disappointed .... 
many of the suggested courses of action have not been followed. " 

"I learned a lot about the evaluation process and I met and befriended 
some wonderful people, especially the evaluation committee members .... 
My involvement was influential with other principals^because principals 
tend to accept requests associated with other principals. Also, the 
influence and involvement of principals and teachers in the training 
workshop for collecting observational data was invaluable. " 
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"From the beginning I felt that it was important that there be some 
"accountability" to the (implementation) process and therefore I was 
pleased that the evaluation was an integral part of implementation. " 

"I learned a lot about the following: the system process and hierarchy, 
capturing and analyzing valid data, the structure of an evaluation, the 
structure of the program department and the involvement of system 
representatives in the evaluation. " 

"Perhaps I should start by indicating that I am a very strong believer in 
providing opportunity for teacher: to accommodate their "ownership" of 
the particular issue at hand. Only by having their input and a sense that 
they are part of the decision will change occur. I believe that I, along 
with other evaluation team members, were given the opportunity and 
latitude to effect a constructive and meaningful evaluation. " 

"I also detect signals ... that the action plan was laid on from "above" 
and that this plan criticized (teachers') credibility-md therefore they 
tended to be on the defensive .... Having said this, slowly but surely 
change is occurring as a direct result of the evaluation. " 
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Findings 2: Questionnaires 

The purpose of this section is to summarize the findings from the questionnaire 
responses and to share some preliminary analyses. 

Characteristics of the Evaluations: Closed Items 

A seven point Likert scale was used to assess six characteristics of evaluations 
(Cousins and Leithwood, 1986). Credibility and relevance of the evaluations 
received the highest overall ratings. The extent to which the findings matched 
expectations, the overall quality of the evaluations and the timeliness of the 
findings for decision-making were also rated highly. The nature, amount and 
quality of communication about the results was given a low rating. 

Utilization was also included in this question as a characteristic of the evaluation. 
This characteristic received the lowest rating. Responses to this item were used 
as the dependent variable, utilization, in subsequent correlation analyses. 

Factors Influencing Utilization: Closed Items 

, Respondents were asked to use a seven point Likert scale to4ndicate the level of 
influence of nineteen items on evaluation utilization. Five factors were identified 
as having a significant correlation with utilization. The two areas having highly 
significant correlations with utilization included the extent to which the findings 
were in agreement with the expectations (r=0.70) and the commitment, 
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involvement and/or resolution of the school district 's Superintendent of Program 
(r-0.83). Three additional factors that had a significant correlation with 
utilization included the overall quality of the evaluation, the nature and quality 
of communication, and the involvement of primary users in the evaluation. 

Evaluation Process: Closed Items 

Based on responses using a five point Likert scale, the process used in previous 
evaluations was rated positively in the following areas: technical assistance 
provided by the school district's researcher, joint responsibility among evaluation 
team members, primary users who learned on the job, primary users who were 
involved in the h nuts and bolts n of the evaluation, and the partnership between 
a trained evaluation person and primary users. In addition, primary users were 
seen to be very supportive of the evaluation process and the goals and activities. 
They also valued the interpretation part of the evaluation process. Lowest ratings 
indicated that the action plan was not well implemented and that primary users 
were rarely involved in developing the action plan, (see footnote) 

Footnote: It should be noted that the action plan is a decision making activity 
and is not part of the formal evaluation process. The evaluation team's 
responsibility includes interpreting the findings and developing suggested courses 
of action (recommendations) based on the findings. It is then the policymaker's 
responsibility to develop and implement an action plan. 
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A factor analysis of the questionnaire items dealing with the evaluation process 
identified mo key factors. The first (with an eigen value of 6.96) included items 
that dealt with activities related to planning and conducting the evaluation. The 
second factor (with an eigen value of 3.67) included items related to the 
development and implementation of the action plan based on the findings of the 
evaluation. 

Involvement of Primary Users: Closed Items 

A five point Liken scale was used to document responses. The findings indicated 
that primary users learned a great deal about the "nuts and bolts" of doing an 
evaluation, viewed their involvement as a valuable staff development process and 
considered the evaluation to be a positive experience. In addition they learned 
more about the program, developed a stronger relationship with the internal 
evaluator as well as at least one other member of the evaluation team, and 
believed that they made a worthwhile contribution to the review process. On the 
other hand, respondents reported few subsequent opportunities to use their new 
found expertise and indicated that they had received limited recognition for their 
efforts. 

Once again factor analysis of those items dealing with the involvement of primary 
users in the evaluation process identified two key factors. The first factor (with 
an eigen value of 4.39) included items related to empowerment issues. The 
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second factor (with an eigen value of 2.94) dealt with staff development issues. 
Rating of Organizational Culture: Closed Items 

Items related to organizational culture were also rated on a five point Liken scale 
and responses were consistently lower than those given to items related to the 
evaluation process and primary users' involvement in the evaluation process. 
There was, however, general agreement that primary users were given support to 
participate in the evaluation, that primary users were motivated to participate, 
and that the evaluation was an integral part of the implementation process. 
Respondents, however, were less inclined to rate the system as being committed 
to organizational learning or inquiry as a way of enhancing learning and 
empowering staff. Furthermore, most primary users were not fully aware of the 
scope of the task before agreeing to participate in a review 

A factor analysis of items also found two key factors. The first (with an eigen 
value of 5.91) included items that specifically described the system's culture as 
it related to evaluation. The second factor (with an eigen value of 2.01) 
encompassed the notion of system support for evaluation. 

Pros and Cons of Using a Participatory Approach: Open Ended 

There was overwhelming support for the quality of the professional development. 

In addition, the empowerment, increased understanding and ownership of the 
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evaluation process were identified as important outcomes. Respondents believed 
that it was important to have involvement of those who were ultimately 
responsible for making resulting changes. The participatory approach was also 
seen as a way of keeping the evaluation practical and honest. On the other hand, 
a great deal of time is required. Also, involvement of primary users does not 
seem to guarantee utilization. Responses highlighted the need for commitment by 
senior administration; the process must be valued and used to inform decision 
making. 

Factors Affecting the Utilization of Evaluation Findings: Open Ended 
Budget was identified several times as a limiting factor, especially in a time of 
limited resources. Several responses mentioned the need for evaluation to be 
seen as an integral part of the decision-making process. This includes a 
commitment to follow through with and implement the action plan. Related to 
these issues is a commitment to the evaluation process and the resulting action 
plan by system and school administrators as well as staff. Responses suggested 
that this commitment is generally lacking in this system. The need for better 
communication - about the evaluation process and the findings - was also 
mentioned. Factors mentioned most often in the responses included: limited 
budget, lack of commitment of senior staff, commitment of key individuals, 
commitment of the evaluation team, lack of system commitment to the action plan 
process. 
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Conditions Likely to Benefit the System and Primary Users: Open Ended 
Several respondents mentioned improving the development and implementation 
process of the action plan. In addition more commitment, support, understanding 
and involvement of senior staff were seen as important. Several persons indicated 
that the system must value and use the results of the evaluation studies. There 
should also be more effort to support primary user release from other duties as 
well as the involvement of primary users in framing and implementing the action 
plan. 

Value of Participatory Evaluation: Open Ended 

Previous evaluations in the school district were generally characterized as being 
highly credible and reliable. In addition, the findings and evaluation quality and 
timeliness were seen as relatively high. On the other hand, evaluations were 
rated less well in terms of communication and time taken to complete the study. 
Utilization was deemed to be generally low. 

General Comments: Open Ended 

There was a number of comments expressing gratitude for the opportunity to be 
involved and stressing the positive nature of the experience. Comments were very 
supportive of the evaluation process. There was, however, concern about the lack 
of or inappropriateness of follow up activities, especially related to the action 
plan. 
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Preliminary Analysis of Relations Among Key Variables 

Using the review of the literature, open ended responses, the content analysis of 
the written correspondence, descriptive analyses of the questionnaire responses 
and the results of the focused group discussion as guides, a number of key factors 
were targeted for further analysis. For example, correlation matrices were 
calculated so as to examine the association among a number of items. 

Several different regression equations were also posited and examined. In these 
cases the dependent variable was the utilization of the evaluation findings, that 
is, the respondents' judgement about the extent to which the findings and 
suggested courses of action were implemented. In the context of this school 
district, utilization involves decisions about the nature and operation of the 
program as well as decisions about program management. It also includes some 
aspects of use as education (Cousins and Leithwood, 1986). 

Statistically significant linear relationships with utilization were found for six 
independent variables: first - the commitment, involvement and/or resolution of 
the school district's Superintendent of Program, second - the quality of 
communication, third - the quality of the review, fourth - the extent to which 
findings agreed with expectations, fifth - the involvement of primary users in the 
evaluation, and sixth - the factor containing several items describing the system's 
culture related to evaluation. In these cases multiple R was 0.72, 0.71, 0.68, 
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0.67, 0.59 and 0.58 respectively; the F value for each regression equation was 
significant at the .01 level. When a regression equation is formed using stepwise 
selection of the above six variables, three independent variables are selected. The 
multiple R is 0.94 and the F value is also significant at the .01 level. 

These initial examinations of relations among variables are limited by the very 
small sample size in the present study. This critical limitation acknowledges the 
possibility of bias and a potentially inflated multiple R. Nevertheless, the results 
tend to support links of these variables with evaluation utilization. 

Findings 3: Focused Group Discussion 
The focused group discussion involved several primary users in a discussion about 
the value and impact of participatory evaluations in this school district. 
Individuals based most of their comments on the specific evaluation and context 
that the, 1 had experienced. Once again, th». value of the review as a professional 
development and a collaborative learning experience was emphasized. The sense 
of competence and empowerment that accrued was characteristic of each 
evaluation project. Without exception there was support for the multiple data 
collection methods and the perceived relevance of the data collected. Primary 
users welcomed the opportunity to participate in the evaluation process and to 
collect data that could be used as the basis for subsequent action. 
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On the other hand, concerns were expressed about a number of issues related to 
the evaluation and, in particular, the development and implementation of the 
resulting action plan. For example, there was concern that the evaluation tends 
to take on a life of its own. With limited resources there was a feeling that the 
evaluation took too much time and required labour intensive involvement. Tfiis 
was especially true when compared to the seemingly limited time, resources and 
energy devoted to implementing follow-up acdvities related to the action plan. 
In addition, there was uncertainty and confusion as to whether the system really 
valued the evaluation process, especially when the findings indicated significant 
(as determined by the evaluation team) areas that required improvement. 
Furthermore, evaluation utilization was seen to be easily influenced by changing 
political and economic circumstances, beyond the control of staff within the 
system. It appeared that policymakers were consistently overwhelmed by other 
matters and repeatedly delayed or proceeded very cautiously, if at all, with an 
action plan. 

Although there was general consensus that the protocol and procedures for doing 
evaluations had progressed significantly, there was also a strong feeling that 
evaluations were still not integrated into the system's way of doing business. It 
was suggested that the participatory nature of evaluation is very much limited by 
structures that are not always supportive of nor in tune with using the results of 
evaluation studies. 



Conclusion 

The present research study provides specific feedback on the program evaluation 
model used in this school district. It also reinforces a number of issues identified 
in the literature related to participatory evaluation and evaluation utilization. 

As an initial attempt to come to terms with some of the findings of this research 
study, the following reflective statements are posited. They provide a basis for 
further improvement and refinement of the program evaluation model used in this 
school district. 

* Organization structures and power relations need to support, better 
participatory program evaluation, including evaluation utilization. 
Program evaluation should be integrated into the system's way of 
doing business. 

* It is important to develop a strategy for ensuring that the action 
plan is developed and implemented. 

* The involvement of primary users in the evaluation process results in 
positive staff development, feelings of empowerment, and a sense of 
competence about evaluation issues and procedures. 
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There must be further involvement of primary users in the system 
and follow-up using their expertise acquired during the evaluation. 

Communication about the evaluation process - all stages - must be 
improved and be of the highest quality. 

Vie current climate of political, economic and educational change 
demands prompt and efficient use of time and resources. 

Evaluation must be valued and seen to be valued; this can be 
demonstrated by acting on the findings of the evaluation. 

Efforts should be made to understand better the perceptions and demands 
on policymakers so that the design, conduct and use of evaluations can be 
better planned. 
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